Learning rate and attractor size of the single-layer perceptron

Martin S. Singleton and Alfred W. Hübler
Phys. Rev. E 75, 026704 – Published 26 February 2007

Abstract

We study the simplest possible order one single-layer perceptron with two inputs, using the delta rule with online learning, in order to derive closed form expressions for the mean convergence rates. We investigate the rate of convergence in weight space of the weight vectors corresponding to each of the 14 out of 16 linearly separable rules. These vectors follow zigzagging lines through the piecewise constant vector field to their respective attractors. Based on our studies, we conclude that a single-layer perceptron with N inputs will converge in an average number of steps given by an Nth order polynomial in tl, where t is the threshold, and l is the size of the initial weight distribution. Exact values for these averages are provided for the five linearly separable classes with N=2. We also demonstrate that the learning rate is determined by the attractor size, and that the attractors of a single-layer perceptron with N inputs partition RNRN.

  • Figure
  • Figure
  • Figure
  • Figure
  • Received 12 August 2006

DOI:https://doi.org/10.1103/PhysRevE.75.026704

©2007 American Physical Society

Authors & Affiliations

Martin S. Singleton*

  • Department of Mathematics, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA

Alfred W. Hübler

  • Department of Physics, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA

  • *Electronic address: martin@math.uiuc.edu
  • Permanent address: Center for Complex Systems Research, Department of Physics, 1110 W Green Street, University of Illinois at Urbana-Champaign, IL 61801, USA. Electronic address: a-hubler@uiuc.edu

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand
Issue

Vol. 75, Iss. 2 — February 2007

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review E

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×